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ABSTRACT 

To remain competitive in the international 
marketplace, the United States will have to recommit to a new era of 
educational reform. Students must be pushed to reach higher 
expectations, and the U.S. educational system must find an efficient 
and centralized mechanism to measure national progress in education 
reform. This document descrlt>es the current trends in the development 
of indicators designed to measure educational progress. Several 
specific problems with indicators are targeted, such as: 
accommcKJatlng federal* state, and district information; effectively 
processing this information; and dealing with the excessive cost of 
creating and maintaining a national database. All the recommendations 
for new indicator systems feature some version of an input, process, 
output model that flows from characteristics of the community and the 
population served, through characteristics of the school itself, to 
characteristics of learner outcomes. Student achievement is the 
primary outcome. This document shows several simple models for 
indicator systems and discusses how to transform the models into 
indicators, the indicators into data, and the data into information. 
The last section addresses national projects to develop indicators, 
the burdens and benefits these projects have on local schools, and 
the effects these projects have on the future of education. {30 
references) (LAP) 
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Introduction 



Wiih the publication oiA Nation at Risk (National Commission oo ExceUeiK^e in Education, 1983), 
the public was warned that what America's young pec^le knew and were able to do was seriously 
affecting the United States' al»lity toc(Hiq)ete in the inteniational marke^lace. AnKrican studoits were 
finishing inclose to last place CM international tests of Madcmic achievemenL As a consequence, 
America's economic «igine, we woe fold, was in danger (rf no longer finidiing first Mvich of the 
blame was attributed to the l^tb of Amerka's pid^ schools. Acfaievanait gains made during die 
1960s in the wake of Sfnitnik had beoi squandered during a subseqinnt period of noinimal 
expectaticms and mediocre pafam mice. To remain cmqietitive, die naticmwcwki have to reosmnit to 
a i»w era of educational r^[sm drivra by higher e3q)ectati{ms for what aU students ought tt^ 
do, in fact, achieve. 

Since A Nation at Risk was puUisbed, there has b?en some extensive work, much of it funded by 
the Natbnal SdeiKe Foundation (NSF), to create new and better mdicators the quality of Anxarican 
education to measure d» pn^ress educaticmal reform. Accofding to Shavelscm et aL (1989), 
educatkm indicates are ''single or conqiosite stiwiuidcs that reflect in^ortant aspects of the education 
system." Mostof the work on education indicatora has dealt direcdy with protdenos of monit(»ing 
change in teaching science and niathaDatics,e^)eciaUy in jumra'aijlsemor high schools However, 
the questions that these systems seonbe^ suited to answer are faiilyfinee of science or mathcmatira 
subject matter. The aefca i e , mo^ of the current work on imlicatar systrans ai^pears to be generalizable to 
many areas of die schocd curriculum.^ 

This repon deals widi various aq>ects of researdt on educaticHiindkatnrs. It begins with die 
rescwrees created at the federal tevd and ends widi needs that exist in local schools. Wc will review 
the results that h&vc been reported fitnn several current pi'p^ts to develop naticxial indicators oi the 
quality of edocadon. We will first give an overview of current indkatordeveloimiaiis and present the 
reasons why a national system of education inriicatnrs is nccesary. Second, we will discuss nxxtels 
for indicates' systons, and die txansfonmtion of: models into indicators, indicators into data, and data 
into infbimation for polkymaking. HnaUy, we wiU look at the i»x>q>ects for develqnng indicators. In 
doing so, we will discuss tl» national pro^iects to devel(^ indicators, the burdois and benefits £0- 
local schools, and die needs of k>cal districts and schools and ways diey might be better met by work 
already being doiw by indirator projects. 



^ These imxkis do not have moch reievaiice to ekmentaiy schools. MoA of the vmi^Ies thai are contained in ihem deal 
with such thi^K ooorsewofk, gradoatkm rates, and teacher jyorfaliiMioo in the area in which they teach-vari^des that 
pertain only 10 secondary levels (tf schooling. 



Overview of Currait Indicator Devefopments 



Cuxrent wotk on educatkm indicats»^ responds mostly to a need to provide information to federal 
agency pdkTmakers. States canboi^directly because oi» of the major concerns in designs for 
statistical samfriing at the national level is eventually to have inftnoatkm that can be ocmipared across 
states. Thmfore, the technology that is evcdving to acoxnioodate state-by-staie ccsnparisons will 
undoubtedly gexwrate indicatore that are useful to pc^cymakers in the indivkiual states being 
compar ed. S(m)c needs (tf stale pcdicynaaken will iioc be oaet by a national systnn ediKaticm 
indk»tors. Natkwal designs for statistical sanyling cunendy are not broad enough to allow states to 
compare districts or schofds. Needs for infmmation common to many states are likely to be 
addressed. However, other needs tied to ^)edal coiHxntratkms students or qiedal drcumstances 
creaied by geography or econcnnics oondititms wiU require infonnation resources to be developed by 
indivichial states. 

A problem that has not bMO adibessed adequately is the fact that local disricts and schools are 
unlikely to recdve many direct benefits torn infonnatioo that will be generated by national systems of 
education indicators. Developers of natioial system hav^ expressed couccxns that statistics reported 
nationally, incUuiing state-by-state comparisons, shouki be uodostandable to a wide audience of 
pdkymakers. But needs of pdkymakers at bcal sdiods and districts have iKit rccdved much 
attoiticm. Energy this far is going into ^v^qanent of a technology that will provvle valid 
infonnation federal policymakers can refy upon. In^Ucatkms diis work for local districts and 
schools have yet to be ttonght through in mndiitoiL 

Part of tliis problem has to do widi stnkrthnlriing and owmrshq). Few tocalschodsOT districts 
wiU have ixiuch direct knowledge about iiattonal indicators or the significance of tow they 
progress in tt:hieving the nation's goals and objectives. Some local sdKX>ls and districts will 
partidpate in collecting data fcgnatMwal indicator systans, but the data they generate 
transformed into infmnation they can use. Local sdiools and distrtets will not know enough about the 
d^gn of lbs indicator systsn to piA the infiiuiaiioo they are ooltecting into a context diey can 
interpret Inadditicm, nooiteatahigherleveiislikdy tointerfaetthedatafortfaem. 

Costisanc^herpartof theproblon. Many indicaitasideittiiled as parted a national system will not 
be phased into a national system for data collection until ways can be kloitified to fumish them with 
dataatBnaccq)taUeco^ In partknilar, indicaiors that require fairly precise accxMntings of te«:her and 
stu(knt engagemrats in the classroom over an extended period are likely to be put aside for data 
collection plans. The time tiiat teachers, students, and administrators woukl have to sp^ creating and 
maintaining diis pan of a national database seons excessive hi light of any benefits they wouki receive. 
Some of the perceived costs oi dus kind €i data collection couki be reduced if ways could be devised 
to make the data collected by local districts and schods serve thdr own needs, as well as the needs of 
technicians who maintain natic»al databases and those of state and national policymaking leaders. 



Local distrkrts ami schools do have needs for infbnimti<m that will suppoit local pc^icymakuig. Some 
of tte chaiacteristk:s identic with ''good" indicators for a natifmal system invc^ pr»:tical 
ccmstderations that help to keep the infonsation from indicators foct»ed m ckdsioninaking about 
school practkes. As a result, the systems of national indicators that have beoi proposed have a 
gmmding in schod practice that makes than rdevant to the needs of policy and governance at local 
districts and ev«i schocd sites. 

The policy questicnis associated with natiraal indicator seem like the kinds of policy questicms that 
managers of local schools and districts also would like to answer. Ifowever, the data currently c(»ning 
from natioaal indkator projects provide veiy litde polky information relevant to k>cal schools. 
Aggregations of data in national reports tend to focus on the nation as a whole and, to some extent, on 
comparisons across individual states. 

The research going into (tevdcqm:^ of a natioial system of indicaicas may, towever, be useful to 
local districts and schods in generating their own data about edi^ationindkaiors. Local districts and 
local schocds do have a growing capacity to satisfy more (tf didrown needs fca- information to suj^xst 
polky and govemarce. Many kx»l sctools have very good tedmobgy and easy-to-use software for 
datamanagemnit The investment neetfaxl to provide training and staff devebpment that will greatly 
enhance the c^>adty<rf sites to gCTcrate and maintain databases of thdr own is nKxiest Woikon 
nat&xud indicator systesos can hdp, e^jedaUy insolv as it ittentifies module indicators t^ 
districts and schools can adopt for their own use. Just as impcntant, woik on national systems 
prcnxdses to yield a moctel aiul technology, including surveys and sampling dedgns, that may be 
adaptable to a tevd of efibn in data cdlectioa ilmt tocal distrkts and scfaods ooukl inaintaiiL 
Characteristics inttinsk to good educati<»iindicatnrs--characteristicstfam define a range of questions 
that iridicators are intented to answei^iiay lead to syndieses erf existing knowledge and fiiidm 
new research tiiat are quite relevant to proMcms associated with policymaking in local districts and 
schools. Questions about schooling effects addressed by indicators in national systems do appear to 
be the kinds of questicms local districts and sctods also need to answo*. 

Why a National System of Educ/itioa Indicators Is Needed 

The failing grade that A Nation at Risk (National Commisaon on Excellence in EdiK:ation, 1983) 
gave to public schools sent a noessage, tend clearly in the Amerkan mainstream, that educ a tion 
needed to be fixed. Another message, h&ad mainly by federal policymakers, was that the corpus of 
information about the quality d edi^ation to all youth was not very useful Seklen (1988) repoits that: 

Tht National (Commission on Excellence in Education, which had 
been char^ with devebping that report, and the critics and commentators 
who followed had to struggle to obtain basic facts about die Iseadth and 
quality of schooling in the U.S. For example, the Commission on 
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ExceUo^ hml to ccMiduct a ^)ecial stutW transcripts to find out what 
courses stiKiaits took in high sdiooL htoidxis of the commission had 
ported, quite ieas(mal^, «at downturns in acfaieveni»t migjtt te 
fewer course m^uhtments in ctm sulyjects, but solid inft»ipation to 
sub^uitiaie ordi^sove that propositicm was iK^*-and still is ncH— ^dlected 
regulaily. (p. 492). 

The need for better inf(Hmation abom natkmal {ffogress in in^iovi^ 
the Nati(^ Scioice Boaxd Oommissicm cm PreooUege Education in Ma t hem a t i c s, Sciei»» and 
Techm>iogy (1983, p. 12) when it lecommemted that the federal goveromem ... maintain a national 
mechanism to measure student acfaievoiKnt and participation tiiat allows national, state, axsi local 
evaluation ami caxiMuisfK) of educaticmalim^ress.*' Since indivklual stales are prinuoily reqxmsibte 
for q)eradng the nation's systCT pul^ schoc^ most of die data availat^ abcnit the status ^ 
edoCTticw natk»wide originates from data coUcctioo pit^^ram s that staira maintain A nwre centralized 
mechanism is needed because variaWcs that define the structure <rf what is observed about the process 
a»i outcon^ of education are quite dififerrat from om ^ate to a2K)ther. In cas» where different s^es 
do teral to lock at the same variables, tte context in whidi data axe ooltected varies to such an extent 
that aggrcgaticMi of infbrmatbn acn^ Aam is oftrai meaningless. At its most t^sic level, any effort to 
measure the health of the nation's education system requires infoimaticm from all states on what 
students have achieved. Cunendy, 47 states have some fonn of statewide testing program (Education 
Week, April 10, 1991). However, tests that are used, grades and objects that are tested, and 
guidelines for testing some students and not testing others varies widely from state to state. 

If achievenaem data from different states are hard to aggregate in ways that show national progress, 
they are equally hard to disaggrepue in ways diat show fiiir comparisom of oat state against another, 
even when the same test is givoi in difiierent states. In 1984, the U.S.Dq»rtment of Education began 
its puUication oS Suae Education StatistUs (U.S. Dq>artment of Educati o n, 1984). This rqxnrt, 
better known as the ''WaU Chart,** ranked states on the average scores from the Scholastic Aptitude 
Test (SAT) and die Anierican Co^ege Te^g (ACT) program, tests that are wkkly used as part of ^ 
sdecdcm process for admitting studmts to many of donation's colleges and universitbs. These 
Cixnparisms were widely crit^ind widtin the research community because of extreme variations in the 
student population who took diese tests in different states. In fact, publication of these conqyariscms in 
die "Wall Chan" has helped to oiergize effcms to design better indicates of national progress, 
induding indkatorsctf studoit achievement diat will lead to mrae fair comparisons of schooling 
outcomes across stares. 

Indicator Systems 

This section of die repent discusses models for indicator systnns and how to transform the models into 
indicates^, die indicators into data, and the data, into useful infcmnaticm. 



I 

I 

S Models for Indicator Systems 



All of the recomnKndations for new indicator systems feature socas version of an input-process > 
output model that flows finom charactoistics of the ocKnmunity and the peculation served, through 
characteristic* of the scho(d itself, to characteristics of teamer outcomes. Sn^imt achievement is the 
primary outcome in all of the indicator models we reviewed. However, other variables, sudi as 
enrollnirat in advanced courses and occupational or career chc^ces tnade after graduation, axe 
considoed. 

A very simple model of schooling (Bgure 1) was used by Raizoi and Jones (1985, p.l2) to select 
the National Research Council's (^n(Q set of educati<» indicator ]Besrated in its first rqxi^ The 
NRCmoddispaiticulariy ^gnificant, partly because it imeded reports of other naodel building on 
iidicatis- systems by at least a year. Two additional variables, expmditures and public attitudes, were 
consitoed by the NRC committee. They were ncK included in die selecticm of iiKlii^tt^ nuinly 
because the c ommi ttee could find no strong research-based relationships between these variables and 
schooling outcomes. 



Figure 1 

National Research Council Model 
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A later version of die NRC repot (Muniai» &. Raizen, 1988, pp. 143-151) considered "Indicators 
of Bnandal and Leadership Siq^xnt." Two indicators were recommended: level federal financial 
support for science and matbemadcs and math»nadcs education and consnitment of r^urces by 
scientific bodies for improvement of mathematics and science education in the schools. However, they 
were recommended as supplements to another Hst of sevoi indicators, which woe given highest 
pricvity. 

Lata* models were more extensive and included considerably more detail For example, RAND's 
basic moctel of schooling (Shavelson et al., 1987, p. vi) adds details at both ends of the Input- 
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Process-Output model as shown in Bgure 2. Outputs from this modd include tacxt dian student 
acMemnent (altlKnigh it would likely be the primaiy output), and inputs include fiscal resouxces 
and student bad^groumi, along with tether quality. 



Figure! 

Schooling Components Included in the RAND Model 
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A nK3ie recent oodel (NFES, 1990) xtfieciing oxisensus finom a broad cross section of education 
policymakers in state aiMl fedend a^ncies describes an education statistics system that covers foi r 
domains and sevoai subdomains as du>wn bek>w: 

I. Studaitandc(»nmunityb^kgroundstatistk:$ 
IL Education resource statistics 
Fiscal resources 

Human and ncmhuman resources 
m. School process statistbs 

Inqi^emented cuniculum 

Teaching quaUty 

School eovironmoit 
IV. Student {nitt:ome statistics 

Student a(djievement 

Student participation and progression 

Student status after high school 

Studoit atdOKte and aspirations 

Other models fdlow an input-process-output, but include spedsl details that reflect particular 
concerns of their designers. For example. Hall et aL (1985) singled out educative difficulties 
(pupil's capabilities, motivations, handicaps, English language facility, out-of-school supports, etc.) 
as one of three background variables because " ... pupils who enroll in some schools enter with 
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cognitive accomplishments and ca|»bilittts, motivadons, and out-of-school environment ami 
resources, which malce educative eSons easier and less complex than those in other schools." (p. 9). 
However, Romberg (1987) specifically omitted instructional time because it seemed to emphasize the 
imponance of managing instnicticm in a way that was not entirely consistent with reforms that are 
being advocated in how mathematics is taught Moreover, the Wisconsin moddccsiverges on a single 
outCOToe (a amseqt^t variaUe) called Junker enrollment, which gives a high priority to infnmarion 
that would show whether cr not studrats are continuing to enroll in most advanced mathematics 
courses. 

The School Refonn Ass^sn^t (SRA) project (McDonndl et aL, 1990) designed a model for 
identifying courseworic indicaiois that wouU fit within a laiger input~proc^^ 
schooling process. One objective was to probe the din^nsions what it means to measure course 
content that goes b^^o^ a title or even a s^labus. Another c4:9ec^ive was to gauge die feasibility of 
actually obtaining in-dq)th infismaticm about oours ewo ik tpdetanrine the quality of infbnnation that 
might be produced using relatively low-co^ methods such as teacher surveys. The project's model for 
student oourseworic iiK;luited four basic elonents (p. vi): 

• Tqsic Coverage 

• InsizucUunal Stnttegies 

• CuiricularOb}ective$(e.g., emphasis on concepts and processes in coixq>arison to b^ 
skills) 

• Teacher Qualificati{»s 

A larger model of the schooling process was m>t specaScd as such, but indicators identified by the 
pro^ could easily fit within several of the models abeady revkwed (e.g., Raizen & Jones, 1986; 
Shavelson et al., 1987). 

IHmer (1991) outlines a strategy for obtaining natiimal data 00 courseworic content and ot^ 
indicators of school processes following what is essentially the RAND model identified by Shavelscm 
et aL (1988). His rationale is that {socess indicators are meded to describe (^jpcstunities that schools 
provide for students to learo. The q>portunities, he maintains, are the most direct results of school 
policy decisions and are the mediators of indirect outcomes representing what students actually do 
learn. Indicators oS school precedes also are iiiq)«tant in monittmng substantive eff<^ to refomni 
schools and in helping to explain student output, such as achieven^ts in mathematics or science. He 
recomm^ids initial concentration ci effnts to o(ritect schod process data on English aiul mathematics 
at grades K-5, 6-8, and 9-12. To control costs, Porto* recommends the use of teacher questionnaires 
about tqncs that students have covered. More direct methods fcH* obtaining data, such as classroom 
obsovations or teacher interviews, are n;Kaee?q)enave and m(»e^)propriate for research studies 
rather than an ongoing collection of statistics for education indicators* 
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The Organizatkm for EcxmomK Ccx>pmion and Deveb|m^ 
intematioiud uxxid for imlicatDrs that genets 

(Bottani, 1991). OECDodsts to foster eoomxmc growth anKiig its 24 meixibcrc^ Education 
indicators are meded to go along with other bette^estabUshed indicata-i in eoonom 
health, sdoice, and technology used by OECD in analyzing txends and making pro^cti(ms about 
ecoiKmnc health and future gxDwth. OECD has created a Center ftg Educatiroial Research and 
Innovation that has been winking since November 1987 to assess interest across nations in having 
education iiidicators and to detennine the feasibility ctf actuaUy generating education usable st 
The OECD mxid curmitly covers six draiains: 

• Student Achievement 

• Education and Labor Market Participation 

• Features of Schools and Schocd Systems 

• Attitudes and Expectaticms 

• Student Flows 

• Costs and Resources 

Three domains— costs and resources, features of scIkwIs and school systems, and student 
adoevement-are iiwluded in the input-process-outcome models cS natitmal indtcatOT systems being 
devetoped for the United Stat^ OdMriknu^m are more unique to die intere^ OECD in 
conductir j comparisons across the nation. They alsoseantorefleaaiatoccupatiCHi by model 
buiklers with tte statistics that are most coaqndble widi the unkpiei^ss oi potitics, culture, and 
technology across nati(»s. Educatk»i and labor maxk^participatkm includes indirators, such as 
**training schemes for 15-24-year-olds loddng for a^** and **relative earnings of empbyed adults by 
educational attainment" Attitutks and expectations wouU consist (Vindicators dealing with data firom 
national surveys of teachm studrats, parents, aiid pul^ (qrinion pedis that deaU with such t^ 
curriculum, equality, qjending for educatkm, and teacher ejqiectaticms. Student flows is a domain that 
deals exclusively with the progresaon of students through various edinration levels roughly 
con^arable across natic»is. 

Transfonnbig Models into Indicators 

Models of sdiooling are the platforms for generating indicators dtat will provicte pdicymakers at 
various levels with a pctart of what is happening and whether things are getting better worse. 
Some reports on the design of indicators have stated explicit criteria fa* going from model to indicator 
(Shavelson et aL, 1987; Blank & DaUdlic, 1990; McDonnell et al, 1990). Other repcsts allude to 
general principles followed in identifying indicators (Raizen & Jones, 198.^; Blank, 1986), but do not 
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idaidfy criteria as adi^inct sttp in ti^ pioc«s that was followed in indtcaKirctesign and 
devekqmient^. 

One of the most important quaUties ctf an education indk:ator is that its de^ 
imprecision of the Gcmtext in which it wiU be inteipietBda^ Oakes (1986) has (Ascribed an 
ediKatkmal imi^ior as a static that reveals ff(^mgt * *i " g aboitt the perfonnance or heaidi d die 
educational system. '*C%viously, indkators do not tell everything about a system. Instead^they 
provide an 'at a glance' indication of ctBrent comiitions and may evoi augur future prospects." (p.l). 
Indica&xs are ^qm^siate for mcmitoringo^er dine the perfonna^ systens that incliKie 

many intsnela^ components. Indicators are statist^ but not all statistics are indicates. Fora 
statistic or measure to be used as an indicator, itmust have 8refinencep(»nt so that ajud^nent can be 
made on whether the condition being described is getting b^ter or worse (Mimane & Raizen, 1988* p. 
28). 

Indk»]ors describe c«utitions that rdlect linkages amcmg differem elements a^ 
system, but they do not allow us to isolate causes. They are paiticnlsriy useful in social simatims 
where dianges can be obsoved with some predricn and regularity, but the control of situations 
needed to infer causality is not feasitde. The raficmale for Wisconsin's mathematics mmitcffing center 
was developed around what Romberg (1987) and Romberg and Smith (1987) describe as a causal 
model. However, Shavels(m,<>akes, and Carey (1987) took issue widi the logic of obtaining causal 
inferences from a monitraing syston intended to reflect the broad range of om^lexities present in die 
schooling process: 

Causal claims fixan ncmejqwiimentalrcseardi must, d i»cesaty, 
r^ (m strong dieorMkal grounds to nik out plausible coumerintexpretadons 
to die proposed causal iumptetati oiL A stn^ the(sy, one that is logicallv 
cona^em and oiq^ikaUy justifiable, q)ecite the oooqxments a cau^ 
system and d^ causal <»aering, as Ronteg points out In die absence of 
stnmg dieosy, we run the risk of inaccurately ^pedfymg die causal noodel 
by omitting oonyoMPts dim OTrequirBd to nde out coonteii uic i'p ietatto ns 
<ffbyincoRectWq)ecifyingdieejdsdi^(and^dirBCtioaf^cansaHty. The 
consequence c» a weak dieory is diat we mqr eniHieously infier causal 
relatkMis where ttey do not exist Of where the causal flow is in the opposite 
directkm ... a national monitiHing syston, oi necessity must casi such a 
wkte net to reftect die "healdi' of maibematl^ education diat it cannot 
possiUv indudetn its specification die level of detail diatwoukl permit 
causal inferences, (p. 96) 



2HanetaL(198S) did not identify qjcdficindkalon^oidy a 11^^ Tbdrview 
was diatq)ecificd^ items (indicatocs)sbODld be idemified by die Center C CTi sti n Ka K ii e «. "~iftlw 

system is to efiecdvety meet the needs of data iiseis'0>- 4) NadoulPonira on Education Statistics (1990) ideiuined 
princip te sd>atgrikmcdinsettmgttpanewdaiasy«Kn,b<tttfaeyhidiiK»»to 
with Klecdffii of indicattn. 
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After reviewing the Uterature on tl» social indicator iiK>veiiient that developed in this country 
during the late 1960s and early 1970$, ShaveLsoo et aL (1988) concluded that indicators cannot do 
some things: 

• Set goals and jsicmt^ Educatkmal goals and pricaities are established by the public 
through its elected representatives. The information gei^rated by an indicattH- system can 
infionn tlK)se objectives, but it is just cs» factor among many in shaping (fecisions about 
pc^icy preferences and piicaities. 

• Evaluatr- programs. Social indicator cannot substitute fis* a wdl-<kagned, in-depth social 
prognun evaluation. They do not jaovide the level <^i»cessaiy rigor or (tetaiL 

• Develop a balance sheet. Sodaliodbators lade the comnion referent availaMe to economic 
indicates ...ediK»ticHi cannot put e^ of its constructs on a common dollar nmric. (p. 8) 

An indicates should be aUe to do many things to inform policymaking. In design of an indicator 
systnn, they are the criteria by which indicators should be selected The RAND pio^ used this 
noodel of sclKX>ling^ and (tevel(q>ed an octensi^ set of critoia to generate actual iiulM^itors (Shavelson 
et al., 1987): 

• Provide information that describes central feamres of the educational system, such as 
teachers' woric \osd or curriculum offerings. 

• Prov^irifarii»tBm abcRitcurremor|ioiieiiiia/^^ 

• Describe educational comlitkMs of particular concern to pdicymakers and amenable to 
change by policy decisions. 

• Measure obsarved befuivior, rather than perceptions. 

• Provide analytical links among important components. 

• Generate data from measures gerierallyaccqyted us am/ re/ioZy/e. 

• Provi(te information that can be imfi/y undlemt>0d ^ 

• ^e/ea5(^/f in toms of timeUnesSt cost, and expertise. Indicator data need to be produced 
within a tin^ frame that is conqntible with policymakers' decision cycles and within given 
cost coistraiiits; they also shoukl be colteciable, aiialyzaMe, aiul reprataUe within currrait 
levels of expertise. 

Indicators shoukl measure enduring features of the system, such as the number of matlwmatics 
courses required for high school graduation, rather than specific policies such as whether computer 



^hlM RA>n} pra^ ccmsulend sev»id alieniadves for devel^^ 

data thai war already collected al ihe ^ate and federal levels would be used. 
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mash is reqimed. They also should noeasure ut»qiiitous features of schooling-Kiimensions that can be 
found in some fcam thxoughout the system (Oakes, 1986, p. 2). 

ShavelsoQ et al. (1987) make a distinctio& between an indicate^' and an indicator system. The 
latter, they maintain, is a set of interrelated indicatOTS based <m a conceptual model (p. 9). Interrelated 
indicatc»s are wcdtd to matdi the underiying ora^exity of interacdoos amcmg shutent characteristk^s, 
teacher qualificatiois, ccmditions of schooling, and c^ier factors. Tliose ccmplex interactions underiie 
major changes obsoved in what students opt to do, such as oiroll in higher level mathonatica courses, 
aiKi how they peifw uL 

Raizen and Jones (1985) (kscxibed a simple approach to devek^nng the NRC model in which the 
selection of indicaicas wouki be "gmded by relevance to poUcy" (p. 29). The SRA pro^ (McDonnell 
et al, 1990), in probing the (tepths of high school ccnirsework, identified four specific criteria that this 
kind of indicator of curriculum process^ must satisfy: 

• It must be linked to a larger model of the sctooling process. 

• It naist di£ferentiate amcsig tracks or levels oi tl» currknilunL 

• It nmst distinguish bet^K«aitl»cunicdum as it is inteiuied by designers ami poUcymakers 
and as it is actually inqilemaited in schools and dassroCTOS. 

• It must measure, to die detent po^Ue, tt» (tepth of the curriculum, as well as its Inreadth. 

The State Eduoidcm Assessment Center, in devebping the Couixnl of Chief Sate Scho(^ (DfiBcers 
((XSSO) model, first ktentified a ^ of ''i(teal"indtcatois (Blank, 1986) in which selection woukl be 
based on a synthesis of results finom wodc on the NRC and RAND models, the Wisconnn monitoring 
system (Rcmib^, 1987), ami otho'isojects alxe»ly under way. From that set. Blank and DaUdlic 
(199()) reported that a list of "prkxity" indicators were sekcted based (m three critoi^^ (a) importance 
and utility of an indicator at national and state levels; (b) technical quality of data that can be obtained; 
and (c) feaability of obtaining required data (p. 7). 

The OECD ooocbl of indicators (Bottani, 1991) fieaturK a teii^late fin- standanlizing desc^ 
individual indicators across the model's six dcxnains^. The various desoiptors in the template inclmte: 

Generalities 

• nameof imiicatoi' 

• definition 

• latumale and relevance 

• pl»» in organizing frame (domain) 



^ The temiriate for standardized description of indicatars is pat of a handbook of international indicators that OECD plans 
todevelq) in '991-92. 
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Statistics 

• basic datE 

• breaks bwns (e.g., by (kmographic variables) 

• calculation tcnqilate (fmnulas) 
Availability 

• technical validity (how likely is it that satistics will be ccnnparaUe across nations) 

• feasibility (difiQcnhws tfiat nmy be et^mtered in crfitaining data because of diffeicnces in the 
way ediK»tion systems are organized in different nadons) 

• political acceptabihty(tlw value that dififinemcoun^KS are likely to place on an indicate 

feasibility and cost effectiveness are comxms that have ^laped most of the cunent work on 
education indicates. AnothCTCoiK^ is the problem of integrating fetkral and state i^eds for 
indicator data, partly to speed up the {access of getting indicator systons <^)erating and partly to reduce 
burdens on states and local schools. The primary purpose of the UCLA/CSE study (Burstein et at, 
1985) was to detemiine the feaalsHty foshic^uiig a national syston <rf educational ind 
existing state-level data on studoit Mdiievemem. Oaeci five altenotivesfra-develqring a national' 
imiicator system conskkred in the RAND project (Shavdstm et aU 19S7) was a low-c^ 
approach where cmly existing state and federal data would be retrofitted to a national model The 
Gesso's Stale Edi^atii» Assessment OentCT, which sponscned the State Science and Mathematics 
Project (Blank & DaUdlic, 1990), was estat^ished hi 1985 with a continuing re^nsil^ty to 
coordinate the develqmient, analysis, and use state-level data^ Most recently, the NFES (1990) 
has been cstaMished as the principal federal mechanism to satisfy a congressioi^al mandate that state 
ami federal agoides cocqjerate in resfaqnng die natkm's elonentary and seosuiary data systeuL 

Actual lists of iiidicators vary in detail and length from seven 'Icey'* indicators and six 
"suppl emen t a ry" indicators in tl» NRC system (Mumane & Raizen, 1987, pp. 2-4), shown in Table 
1, to ahnost 40 indicators in RAND's model (Shavelson et al., 1987, p.37) of a "piggyback"^ 
indicator system as shown in Bgure 3. 



^Iii&ct,CXI$^s 1990itpoct(m$taieiodfc^ani8repKse«edasa "first ever'craiirilatton of ^ate-by-state data that 

have been integntted to Ht a angle modd for kwldng at the oxKfi^ 

iiatk»widB(p.2). 

^The "{Hggyback" sy^em is cme diat expands conem dtfa ccrftoctk» elT^ 
Ednc^onal Ph^fcss. Other optiooscmisideied hi the RAND jHcyect would iim)lvet^ 
collectim ctf dM indqmutou of what 0 beii^ (tone now by oAer agencies. 
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Table 1 

Indicators in NRC System 



Primary Indioitm^ Supplen^tary Indicators 



Extent of studmt learning in 
mathffmatics and science 
Extent of «jentific ani! mathrn^tk^l 
literacy of ufailts 



Enndfanent data for mathematics and 
sdoic^ courses takm by studoits in 
lu^ school and the time spoit on the 
study of science and matibcmatics in 
etementaiy and middle^unior high 
schools 

Nature of student activities during 
science and madiematics in^nicticxi 
Extent 0f teachers' knowledge in the 
subject matter they are expeoed to 
tea^ 

Salaries ^aid to college graduates, 
with particular sub^-matier' 
spedjUdes, who choose to enter 
various occupaticHis 

Quality of die curriculum content in 
state guiddines, textbooks and 
associated maiarials, tests, and actual 
classroom infraction in sciei^ and 
mathematics tigpugh gMtcfaing to 
exemplary cmricnnim Irameworks 
along four dimensions: breadth and 
dqitn of tre atm e nt and scientific and 
l»dagogic soundness 

The NFES (1990) did not identify indicators, as such. Instead, broad recommendations were 
made under each category of its indicator modeU described eaiiier, for various kinds of data collection 
and r^xjrting that ^ouki be undertaken by different federal agaides. Some recommendations deal 
with improved use of data already known to exist at the state or federal leveL Otho* recommendations 
involve disaggregations of existing data by such variables as sex, ethnic group, language status, 
ccnnmunity wealth, and family income. The NFES also recommends collection of some new data not 
availaUe now. 



AtxKJunt time asent on scknoe 
and mathematics ttomewosk 

Teacher preparation-college courses 
in mnthrrnatics and sconce, majors 
ami minors, advarced degrees 

Teachers' use (rf time outside the 
classroom spent on pofessional 
activities relited to the teadting of 
mati^natics and science 

Materials, facilities, and sillies 
available and used by teadiers in 
matiiemattRS and s^ci^e instniction 

Level federal finmidal support 
fior sdenoe and matfaematks 
education 

Coiiimiiment of resmroes by 
sdoitific bodies for die 
inqHPvemeot of mathematics and 
sdence educatkm in die schods 
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Figures 

Indicators in RAND "Piggyback" Indicator System 



& 




itures 
Pdcent ctf pearscmal 
inoonie expciKted d 
education 
Beginning teacher 
salary 

Averse trad ie r 
salary 

Class size/traching 
toad 

Computer use and 
laboiBtofy facility 
Resource adequacy 
Computers available 
at the school 
Experienced teachers' 
salaries 



Charaiaerigcs 

Course- taking 
fcquirempits 
Teacher idanning 
tuoe 

DnxxMitraiK 
Student enrollTnents 



a 

Descriptors 
Experience 
Cooafoit with subject 
matter 

Recency education 
cnxichmmt 

SfiMiSQt 
Q'aracteristics 

Raoe/^hnicity 
Gender 
Courses taken 
Grades 

Socioeconcmiic status 



•Cwriadum 
Textbook ami 
mttcrialsuse 
Covera^ dccse 
topics 

•Instruction 
Hosoewofk 
Stodaitnseoflabs 
and computers 
Telling mohods 
Access to labs and 
computers 
Assessment 



Mathematics 

of all^udents 
college-bound 
seniors 
jnospective 
soraceAnatfa ma^jcs 



StiMteniPigtiacaflflfl 

ExtTKunicular 
activiti^ 

Oment madi/science 
cmrse-taking 



StuAaitAttinates 
Inteiest, liking, etc. 
Social usefulness 
Career relevance 
Inl«)d«l cdle^ 

exceptions of 
madi/science 



The CCSSO project (Blank &Dalkilic, 1990) identified 1 1 specific indicators, along with 
anticipated data sources as shown in Figure 4: 
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Figure 4 

CCSSO Priority Indicators 

Student achkvexnent (NAEP) 
Stodem attitudes/jntentkais (NAEP) 

InstnartkmlTlme/Pankapation 

Grades 7-12 come auDUiiient (state data cdlected by CCSSO) 
Elementtiy minutes per week (NCES/Staffing Survey [SASS]) 

ChiiTiculmn Content 

Students' oi^xmunity to leam (data not available) 
SdioolCbmlitiQns 

Class Size (SASS or (XSSO state data) 

Number of course pre{>axations per teacher (SASS CCSSO state data) 
Course ofifoings per school (SASS or CCSSO state data) 

Tea::her Quality 

Courses^credits in sdoiceAnath (SASS) 

Teaching assignments by cenificatioo in field/subject (CCSSO state data) 

Gender and race/Bthnidty by student or teacher indicator (CCSSO state data) 

During 1989-90, the CCSSO project pieced'^ tocher state-level data (m all indicators except 
student outcomes^, curriculum content, and number of course preparations per teacher. From its 
own surveys, the project obtained data related to one or more indicators from 46 states. 



Transforming Indicators into Data 



A great deal of mediadng activity takes place bdbre ooc of the indicatoss that we've seen identified as 
an dement of a national imiicatcv systnn beconies a statistic, or, as is cixxe c^ten tte case, beconxs an 
integrated set of several statistics. Tte indicators are really constructs that tend to evoke a "common** 
sense that sonsething is being mrasmed, and that mere of it or less of it will teU us there has been a 
change in the quality of education. A M^ierpa'puial expenditure is sensed as an increase in public 
support for schods, if not a direct inoease in school quality. Higher dn^xmt rates are sensed as a 
decrease in school quality or effectiveness. Indicators arc rarely something that someone can observe 
directly. 



'^Fnm its own »ir^«ys,tl»CXSSOim:t}ea has cHMained data lel^ In 
iheeariyidttsesofthisiOTiectttbeipialityof dataoxnii^fi^ Therefoe, 
comparisons across steles usii^ the CCSSO inodd ffiie godei«andaMy ^laiive. 

^State-hy-staiBdaa win be available fiDTO the National Assessment of Edi^ in June 1991, when 

first state-level lesuhs on n»thenuttto at grade 8 win be rdeas^ 
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To geaoBte data, most all of the "iiKUcators** nmst be transtaned into specific questions or 
"pointers'* that can be answered cr quantified as pan an instrument or souie other souxce for data 
ccdlecticm. Sometinies, the souice is a fife fixmi a district, county, or state ofHcewheie certain kinc^ 
databases, such as certificates held by teachers , tx textbooks odered for instruction, arc maintained 
fiom year to year. In generating data fc^ an indicator, numbers are taken from the file and reccmied 
somewhere else, according to scHoe prc^oo^ that has been w(»ked mt ahead of time, usually with a 
fair amount (^laecs^n. More(tften,thedatacollecticmiiivdvesasurvey,aform,ak)g,oratestthat 
is oraqdeted by students, teacters, administraiors, wcteiks within an individual school en* by an 
external observer who goes to a school site. 

Few indicatois are transfionned into data in isolation fitm other indicators. Moie (rften, the 
qiiestions that define a particular indk»tor are ctmdnned in a survey or form widi qi^tioM 
other ind^atca^ A san[q)ling plan is (tevised, and data for sevoal different indicators are gathered 
simultaiwCTisly. 

The technology for tran^mning indicators into data can be expen^ve, especially if external 
obKTvm must be hired to do data coUectkin at school sites. Ofira, dollar costs are kept within 
acorptaUe bounds by ^lifting sanac d die tHintei iji data odlectkn to local districts ax»l school sites. 
However, the added responsibilities placed on schod pasomel to gather data for somecme else's 
project often increases resistance in local districts and school sites to pcrfOTm the necessary services. 
Sometin»s,schoc^distrkts, and evCT states have no ftovisicm of iiKiicator data is required 

by legislation (»- it may be a oontingoicyfbrieceivingfukialfumis. Nevertheless, the real dcdlar 
costs associafed with the kiiKl of extensive data ooitoctioo needed to siqypcst a national system 
education indicats»s appears to stimulate a search for efi&kscy of ^fort 

Pands engaged in designs for a national system (tf edi»itkm indicfiiors have aU leotmBDended 
extensive use of existing sources <rf data rather than devdoping new data ooltectitm pntgFBins. 
Shavelson et al. (1987) coosidersd an all new techmdogy for datacollecticm as one of several 
ahonatives, butrecommmded a "piggyback" tedmok>gy that combined some new data collecticm with 
the use of data from natiooal surveys and other existing soorees. Mumane and Raizen (1988) and the 
NFES ( 1990) also recommended extensive use oi existing data, but were quite explicit about new data 
that would eventually be needed. 

The fim priority in developing national indicators is to provkk a nationwide picture of education. 
It is followed dosely by a desire to be al^ to inake state-by-state ccmqiarisons iia w 
techiiically valid and meaningful to policymakos at various levels. The NRC model (Mumam & 
Raizen, 1988), RAND model (Shavdson et aL, 1987), CCSSO model (Blank & DaUdlic, 1990), and 
the woric the NFES (1990) aU reflect ddiboatkms on the need to do state-l^-state coniparisons. 
In the first phases of data collection for a system of national indicators, the probaUe sources of 
costing data are roostiy nationw^ education surveys already being conducted by the National Cditer 
for Education Statistics, including: 
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• Natkmal Educatk>nal L(»igitudinal Survey (NELS) (some states) 

• National Assessment of Edis»tional Progress (NAEF) (37 states in 1991 ) 

• C^»iai»nCore<tf Data Surveys (CXD) (all states) 

• Schools and Staffing Survey (SASS) (all states) 

In one way (s- anotber, these surveys would be modified and expanded to allow for new linkages 
in infonnadon called for in the ii^t-inocess-outpat models that various indicators panels have 
produced. 

Other possible data sources listed by the NFES (1990) include: additional NCXS data collectioas, 
such as the International Assessnawtt <rf Educational Prpgiess; data collections from othg agencies of 
the U.S. Department Educaticm, siKdi as tte C^Qce of Gvil Ri^ and tte C)ffice Sp 
Education and Rehabilitative Servk^es; and data odlectk»s fn»ntte U.S. Census Bureau and other 
federal agencies outside the U.S. Department Education. 

Data collections that exist in individual starts also are conskleredas possitde sources d data, but 
only the CCSSO {sp^ (Blank & DaUdfic, 1990) seems to have gone voy far in devek^g a 
niethodology for obtaining these data in ways thia wiQ alk>w them to be coinbined or coo^^ 
states. Developers of models for national indicator systems have been quite cautious in r ec ommrnd i n g 
the use of data cdlected by indivUuai states. DiffierBit states ooltect different data under di£feiing 
circumstances that make any aggr^aticm of data Maoss ^tes very difficulL 

Burstein et aL (1985) found that even scoTN {« achievonent tests are di£Bcult to combine in 
meaningful ways KTOss states. While they considered the possibility of statistically equating a limited 
anxnmt of adiwvenient data coming fiom satewide testing pn>grBiDS, aU c^her project 
te«ied to see the NAEP as the prin^ry sonroe of data cm 8tiKi«it p cf f or ma ncc. Until recently, NAEP 
designs for obtaining a nationwkie san^e on studem adiievenient data allowed only for 
disaggregation of data (town as far as U.S. geogrq)hte regions. Conq^rim of data across states will 
be possiUe beginning in 1991 with NAEFs rqxat of natfonwide peifuma nce in mathematics at grade 
8. Even the CCSSO's State Educaticm Assessment Center (Blank & DaUdlk, 1990) plans to use 
NAEP in its assonbiy of data coming from individual states into a angle set of ii^catCHS. 

Oeneration of some indicator (kta already is taking idace. Many of tl» indicators that have been 
induded in models of national systems are assodaied with natscmal surveys condiK^ by the 
Department of Ediu^tion and otl»r fetkral agencies. However, qyeration of anything close to a 
craiplete system of national indk»tors envisioned by HaU (1985) or Shavdson et aL (1988) still is a 
long way off. CC^SSOs State Science and Math Indk;ators prp^ (Blank & DaUdlic, 1990) has made 
some progress in opoating a coonplete system, with its bottom-up approach to assembling state data to 
fit a national frameworlc Some indicators in the CCSSO system use data from NCES national surveys 
and cover all 50 states and tiie District of Columbia. However, the pans of the CCSSO national 
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picture that require information from states thonselves are mis^g because many stains (k> not yet 
paiticipats. Mcmiver, scHiKimiicatorsintteCCSSOsy^emmiuiredaatfaatfkmH^ 
some of the data that do exist are questionable when data faxoa diffomt states are combined or 
ccnnpared. 

TosKkywnniMkU for natkmalindkaton also Cunemly, there are miiir/ good daa 

collection programs, but no coordinated system yet for linking resources, processes, and outconcs in 
the a»xi|nrhensive way that models fix a national systnn call for. Outcomes ineasures are UmitBd. All 
recommendations from all rqxsts we reviewed included the use (tf NAEP data fcH- at least some 
outaRDeindicaiDrs that deal with Mditeven^L But NAEP is limited, both in the number of grade 
levds that it assesses (gF^ 4, 8, aiKl 12) and the abiHty of its sanqriing design, until 
disaggregate infoimatiim by state. By 1992, NAEP plans to have staie-by-state data in mathematics 
for grades 4 and 8. These data will allow fOTscmie direct comparisons <tfstat^ but, given the cuxrnit 
two-year cycle for assessment in mathoxiatics, analysis of time series ^aoss states will not be possible 
in this decade. 

Virtually all Tcpcxts that we reviewed addressed the wed for outcome measures that involve open- 
ended wosk or actual performance by students, ratlMr than studous' respcmses to multiple-choice test 
itoDS. These alternative fomis of a^cssment should be a supptenott to aial, in some cases, a 
replaoenmit for cnnent ways ctf testing that focus (» discrete ddlb and limit student responses to 
contexts that are quite coofii^ TI» tecfanolt^ for this kind of asse^ment is limited except, perhaps, 
forwriting. ^foIeover, the techiK>logy requires a consensus for what students need to know that has 
yet tp be established within contqtt areas. 

Ftocess ittdicaton are constrained by large hc^ in what we are able to observe with adequate 
efGdency. Models for national ^steins caU for indicators <rfc ^ )port uni iy to leam that go weUbeyo^ 
tides ctf high school courses. McDonndl et aL (1990) have competed a feasibility study on analyzing 
c ou r sewo rk by time spent on different topics. However, the ^fort needed to include this kind of 
analysis as a noeaningful pan an ongoing data coUectioo program is daunting The methodotogy tiiat 
was used in the study was designed to fit the dimensicms of a high sdMxH curriculum. Adapting itto 
fit die cuiT^um and instruction of etemmtary schools is a difficult task. In the past, this kind of 
analysis was canied out by loc^g at textbodcs, uoda die assunqnion that teachers taught what 
textbooks presented. That assunqjtion is far less likely to be valkl now than it was in the 1970s and 
early 1980s. Even if it were va]id,indicatcns that teU us iKmweU teachers follow textboc^ are not 
consistent with current standards for how mathematics shouki be taught aiul learned. Porter (1991) 
has proposed a first step in geneniting indicators of currkuhim processes where a sanc^le of teachers 
natksnwide wouki ccm^lete written questionnaires dealing witii topics covered and related variaMes in 
English and mathematics. Oassroom obsCTvations or even die maintenance of day-to-day k>gs by 
teacters would be more senative. But Porter thinks they are too ambhknisfo' the on^nng process of 
gmerating indicators and ixx>re ap^ax^niate for areseareh study. 
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Transforming DaU into Information for Poiicyniaking 



For datt to have usefulness and power, they most be |>aclcaged as infom^^ The tiansfonnatiofl of 
data into infonnatkHi requires soate oonsickraticm about an aud^K^ The same data can be packa^ 
in difiierent ways to satisfy the needs and interests different aiKiiences and the ways in whkh data 
are likely to be used. The priniaiyaudiaice for infiomiatun from naticmalind^^ 
of fedoal policymakers. 

Meeting the needs of a Mtioaal audience for indicator information is deariy a priority in all of the 
malor presets iiwrevkwed(Muma2»&Raizen, 1988; Shavelson ^ al., 1987; Blank & DaUdlic, 
1990; NFES, 1990; Burstein, 1991; Natkmal EdiK^ticm Goals Parol. 1991). The inqxsrtance of 
ccnsidering ind^aior devek»pment in tte ocmteia c^* informaticm and audiences was a 
exfnessed by Mumane and Raizen (1988) and Shavds(» et aL (1987). In both cases, indicators that 
were selected for national nxxtels ostensibly fit some assumptions about policymakers as users ctf 
information. Special consicterations were gjvai,f(7exarn|de, to inclusion of iridicators on race, sex, 
and ethnidty in national irKidels, so that sampling (ksigns for data cdlection could allow for 
disaggregation of outcomes, such as snKl^t achievox^ data by characteristics of studrats. These 
features in the moctel were thought to be partkulariy inqjoitant to concerns about equity amc»g federal 
policymakers. ShavelsonetaL (1987) also expre^edotmcenis that {kvel(q>ers of national systems fn 
education indicators pay |»rticuhr atiendoo to pdicymakers as the primary audience in older to avoid 
problems that had a negative inqi^ on the sodal indicators movonent in the 196QS. Researchers 
became die prinsaiy users of social indicator data during that period, and the infcamation became 
shaped in ways that lessened its usdiilness £nr focteral poUcymakers. 

National reports on indi c a t o r s are being ^danned or r pc oinnTenrie ri in several proycts. TheCXSSO 
(Blank & DaUdlic, 1990) has already [mUisl^ its first report primarily as a set (tf comparisons across 
states f<^ individual indicatcss. Tte National Educatkm Goals Panel (1991) will jmUish its first 
natisMial rqxntcaid ediv^ion itKiicatOfs in SqMember 1991. The Special Study Parol on Edi^ation 
Indicattffs (Burstein, 1991) is recommending that NC!ES publish reports in each of six areas on a 
regular schedule. National surveys, such as NELS and SASS, will presumably continue to publish 
natkxial rqxnts for primarily a national aiKlience, althcmgh many reports will contain data for particular 
states as part of Kate-by^state omqMrisons. NAEP does ha^ a program for {novidingrepons about 
q)eci& states if tte ^ates fond tl» expansion of NAEP data collection and processing necessary to 
gmerate a state r^xst 

Usefiilnessof iiational indicati»s for state policyinaksrs will likely be liimted. Statesaremost 
likely to see information about themselves in publisted repents that show state-by-state comjarisons. 
Profiles (Vindicator data fjyindwkiual states does not seem aniong current rqxws that fecteral;^^ 
will provide. However, states may be able to genoate their own pn^es from databases created by 
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national inJicatCB- {nojects. Shavdscm et al. (1987) recommended diat natkmal indicator systems 
incliKte san^ting c^gns large enough to allow for state levd disaggregation of outcone data by input 
and prnxss variables, such as chaiactexistics(€stu(teDts and qualif!^ However, 
national data cdlectkMis inograms in education, except for NAEP, do not cunently appear to have thl*- 
kind of cqncity. 

McDoimell (1989, pp. 256-257) considered and rejected the possibility of using reptsts from a 
national indicates' system to sc»nehow increase tl» accountal»lity of states making certain kinds of 
improvemen ts over time. Ste reports that policymakera who were interviewed as part of die RAND 
indicator pn>^(ShavelscmetaL, 1987) were unanimous in thdr opinions that a national indicator 
systnn should noi be used to heighten accountability. Moreover, the level of detail that wouki be 
required to use a naticmal indk^ator sy^em to establish linkages betwem specific polides and outcomes 
in an indivulual state requires a huge [nogram for data ct^ection that carries unreasonaMc costs. 
Reports to individual stales, if they are expected tu affect stale polky, wouki need to be fidtoitd to the 
special needs of governors, legidafiors, or chief state school officers and the levels of state education 
policy each group r^scsents. Reports also might need to be taikml to meet unique conditions that 
defii» the cratext for education iQ indivkiual states, and tinied to coordinate with di£f^^ 
decisicmmaking in diffoent states. 

Prospects for Developing Indicators 

This sectioa of the report discusses the national projects to develop indicators, the burdens and benefits 
for local schools, and what the future may be like. 

National Projects to Develop Education Indkatm 

Since 1983, there has been a great deal of work in designing a national system of education indicators. 

Most m^jcn* projects or w(»king groups have dealt in one way CHT another with creation of 

modd that captures the nx>st salient policy-relevant features of schooUng and associated outconoes. A 

large pan of the early work has been funded by the NSF to create indicator systems for science and 

mathematics. 

This early work includes suj^xnt for the GommittBe on Indkatt»s of RrecoUege Science and 
Mathematics Education, sponscaed by the Natkmal Research Council (Raizen & Jones, 1985; Mumane 
& Raizen, 1988), whkh produced one of the earlier models that atten^ts to link school inputs, 
processes, and oulcon»s; RAND CoipOTatkm's indicator project that weighs costs and benefits fw 
ahemativf systems for actually maintaining a system of education indicators (ShavelscHi, McDonnell, 
Oakes, Carey, & Picus, 1987); the State Science and Math Indkators Project, sponsored by the State 
EdiK^aoon Assessment Center and created by the Council of Chief State School Officers (Blank, 1986; 
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Blank & DalkUic, 1990), which has recently produced the first set of state-by-state data tied to an 
indicator system; and the School Mathematics Monitoring Center^ at tte University of Wisconsin- 
Mmlison (Romberg Sc. Stewart, 19S7), which pn^x»ed to track nationwide progrKs in inq>lementing 
new policy actions designed id change the teaddng and teaming of mathonatics in U.S. schot^ 
While these projects and study groups have addressed the need for indicator systans in science and 
mathematics, all have produced models fn* indicator systems that are miaptaUe to other areas of the 
schod cuiriculum, especially in secoiKlaiy schools. 

Odier work also has attnided to designs f(» educati(m indicator systeins and each has produced a 
model of di£femit aspects of tte schoding process wtoe better infionmtioa wmiM lead to better 
pdicy. One involved a** 10-year plan" for data cdlectioncoamnissioned by the U.SDq}artment(^ 
EdiKation as part of a redesign of its Natknial Caiter for Ech^atkm Statistics' elementary and 
secondary data collection program (Hall, Jaeger, Kearney, & Witey, 1985). 

The National Educatim Statistics Agoida Ommiittee (NES AQ of the Natk»al Forum on 
Edtication Statistics, is carrying out a mandate to engage in ooq^erative, consensus buikiing activities 
thai will lead to an agenda for improving the natim's dementary and secondary education statistics 
system. The Naticmal Fonmi iiKltuln nuxe dian l()Orepiesentativesfr<Hn stare and federal educaticm 
agencies and national education oganizations (Natic^ Forum (m Education Statistk;« (NFES), 
1990)10. 

The National Education Goals Pand is nxmitoring goals for the year 2(XX) that woe agreed upon 
by President George Bush and the National Governors' Association {EducaHon Week, March 7, 
1990). The agreement on 6 goals and 21 objectives is the itsdt of commitments made by Bush and 
the nation's gove rnors at an educati«i sucmmt ccmveoed by the Bush administration in the foil of 1989 
in CSiB^mesville, VA. Mechanisms for traddng the {sogress of the nation and the SO states in meeting 
diese goals are bdng developed by a National Education C^»ls Pbi«1 that inclutfes representatives firom 
the National Governors' Association, the Bush Admini$tratk>n, and majority and minority leadm of 
the House and Senate. The pand has jdedged to issue an annual rqxxt card on the implenoentation of 
National Education Goals, beginning in SqMember 1991. During the past year, six resound groups 
have convened to consider each tte 6 education goals and to recotmnend tte kinds of indicators that 
should be included in the Pand's first report (Natiwial Education Goals Panel, 1991). Forthemost 
part, recommendaticms regarding die September 1991 wpcsi have dealt with national sources of 
education statistics that already exist, sudi as the Natk^ Efeahh Interview Survey, administered by 
the Nationd Center of Ifedth Statistics, NAEP, and tlw National Educatk>nal Longitudinal Study 
(NELS), sponsored by the NatttMid Cmter ft^ Education Statistics (NCES). 

^^This iHioject is no longer funded by ibe National Sdeace Fonretation, 

^^^The National Fonnn, established in 1989, i$ the priadpal mfriwnHm for snptement^ goals of the Natimial 
CtoopeiativeEctoattion Statute System, ot^lyy the Hasrfa^^ Thissystnn 
provides a legislative mandate and structure for a fcderal-statB pannexship that coltects and reports eiemem^ and 
seoxxtery education statistics iim)» the «ispic(» of tiw Colter for Sttlisti^ 

21 



26 



These resource ffonps also have {Boposed sweqnng new efiiHts, including a national dau system 
that would ooUect dam for each stucUrnt beginning s(»iettme prior to entiy 
syston of national examinatkms tuat would measure peribmances of aU stiuknt^ 
national standards. Overall, these resource groups have refxsnmcnded tdotc than 50 indicatCH^ and the 
cone^XMtding sources of data that dtber exist now or need to be created in ordo' to generate them. 
Current sources for these indicatK»s are mainly limited to nationwide data and will not allow state-by- 
state tabulations until tl»r sampling designs arc expanded. 

A similar project involve swsk by a Congressionally mandated Special Study Paiwlon 
Educational Inductors, which is charged with advising the ccmmnssioner of the NCES (Burstein, 
1991). The project currendy is woridng cm (teveloping sets of indicatts^s tied to six **enduiing" issues: 

• Knovdedge, skills, attitudes and di^x)sitions for well-educated citizens in a 2 1 st-centuxy 
donocratk society 

• Quality of schools 

• Readii»ss for school 

• Socktalsui^xstforedi^ation 

• Educatiotol contributions to eoc»iomicprcxluctivity 

• Eqdty in (^ypcmunities^experio^es, and faults for children at risk of school and societal 
failure 

Panel membos will {sopose that NCES putdish a series of indicator reports oa a regular schedule. 
Each repot would include current inforaiation on key indicators fw one of the six issues. 

Related effoats include the Schoc^ Refonn Assessment Project, which devebped strategies for 
nx)oitoringd»co9it«tt and quatityctfcoiBsewadc taken by high schocd students (McDon^ Burstein, 
Omtiseth, Caterall, & Moody, 1990), and woric of die OfiRce of Educational Research and 
Impipvemen t (OGRI) State Accountability Group (1988), in whidi recommendations wov made for 
in^nuving indicator systems for establishing accountability within states. In addition, UCLA's Center 
for the Study of Evaluation ccmducted a study in 1984 ami 198S d the status testing systems in 
individual states and the feasilnlity <^ condoning information from existing state tests fen* purposes of 
comparing student perfonnance across states (Burstein, Bakn^, Aschbacher, & K^sling, 1985). 
The major puxpose ctf the developniKit wmk cm indicator systems since 1983 has been to ]m>^^ 
federal and state policymakm with a more conq>letB picture of the effectiveness of America's schools. 
However, as Burstein (1988) describes in his brief history education indk^ators in the 1980*$, 
events have been driven by a" ...stPWig pressure for i mprovemen t, a need fwnieans to assess the 
impact of educaticHial refonns, and c ^iseqiMnt pcditical maneuvering to (ktermine who sets the 
standards and who ircasures i»x>gress toward thon." (p. 78). 



22 



27 



Burdens and Benefits for Local Scfaools 



At a tinie when kx:al schoc^ are assuzning man respoosilslity far site-based management, they are not 
likely to find miK:h infnmaticm from national indicator systems that mil penain directly to their own 
{)olicymaking. Halletal. (1985), for exaniple, were conconedabcMit the impact of indicator systems 
on local districts and schools, but tl^ concern was more for the effort that woukl be requinsd fn>m 
teadiers and administratffls in Older to coUect data than for benefits firtjm naticmal indk^ 
be realized by local Khocds and districts. Extensi>^ sanqding designs woukl be required to allow for 
disaggregation at the level of local districts or schools of outcone measures, siK;h as achievemem^ 
andccmtextvariaUes, suchas/iomeHvrfcasj^f/Kil McDonnell (1989) maintains that the benefits to 
local districts would be limited to infonnasion that links outcomes and processes for types districts 
(p. 254). 

Local school sim are iKH likdy to heir own inagereftoed in any aiai^hot of education diat a 
system of natkmal indicators is likdy to )«7avi(te--not in this decade, at least The hierarchy ctf reports 
that will get established during the 1990s will indwie various kinds of national profiles, ak>ng widi 
comparisons sxKHig individual states. At the very best, fxpesofdistricts within states might be 
described by data ocmung from natfonal indkatOR. An individual district ntight be aUe to identify with 
anotter distrfct that seryra communities widi rimilar background charBCtetistfcs and fi nan c ial 
resources. Any further dis^gregationctf data to types ctfschocds within districts seems improbaUe, at 
least in the fbreseeat^ futme. In additifm^furtlMr disaggregation is not a proq)ect that individual 
sdiools are likely to wait for as tbey (^xrate thdr own programs and mate more of the pdicy that 
affects what they do and what they may be able to aooom|dish. 

Nfany tocai schod sites wiU particulate in the generation of natk^ ittdk»tt^ because the saniip^^ 
derigns to be used for data ooUecticRicaU for die classroom or sectkm of a course--a unit of instruction 
definedbyasetof students and their teacto~to be the unit of conservation. Halletal. (1985), fen* 
esmqde, recommend that the »:tual ga»rBtk» (tf indicators begin widi a coQecticm of what t^ 
n^ro records, wtoe infonnatkm about individual piqrils can be linked to :"fonnatii» about indivkiual 
teachers. In Hnked form, the new units of information from diese iiKiividual records will be 
aggregated to higto* levels that inchideschocd, district, and state. The burden on any single tether or 
group of students will be slight Thoe are many teachers and groups ot students across the country to 
sample from, and dengns for a nationwide sampling of data tend to focus on getting the nM>st 
information from the smallest interventions into the mainstream of school activities. 

The a)lkcti(m of data for indicators may disruin the kIiooI life erf some individual students aiKi 
dieirteaclMsrs, but it does not temi to invade die collective lives of a school or distri^ An acceptable 
balance at the school site is struck between the burden of data collection and die benefit <rf information 
received. No direct benefits are received, but die burden of data collectu>n is voy small. Indivkiual 
teachers are willing to participate as long as the preparatbn that's neected to provide data is quite small 
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ami the infonxiatkm ^at's xequized to OMX^leie surveys or fimxu can in(»Uy be zec^ 
and not researched. 

The hmkn at the schoc^ site of geooating im&ator data will escalate sericnisly when indicators of 
what teachos teach and students »mially have an q^xstunity to leam are seriously pursued. 
Su^ainable pn^xs into sa^ things as qqxxtunity to leam Imve nu^y been put c^. partly bec^uM of 
cost and complexity of data collection. Teacters' inqiressions of topics they have covered (or usually 
cover) during die year and relative entasis diey lave given to each topic may not be accurate. They 
also may not jsovide the kinds of detail needed by natk»al analysts to get a good picture of what all 
students have had an oppcatunity to team. Tliev^bre, scane new kinds of recoid keeping or forms 
maintenance wiU be required finm teachers and k)cal adoxous^^ 

what is taught are to jnovide true images of school curriculum and instroction. Other kinds of record 
keeping such as diose used in the Sctxxd Refom A cessment |aoject (McIXmndl et al., 1990) will be 
necessary to ^neraie indicators for odierdimensi(»s of school context Those incliule the use of 
"hamis (m" methods for teaching science and maifaemadcs, or the am(»mt d real time and effort going 
into problem solving and critical thinking. 

As long as c^gn aiul devdqment d edsaatixm indicators remains a tc^^down pnxxss flowing 
fiiHn the federal gove r nmo it to ituiividual states, the oo^ of generating many indicators needed to fill 
Otttanationalpiaureof what is being acoon^ilislMd may seem unreasonalde. Data that testes and 
administrates wiU provide to niaintain a natiiKia) indk^tor system wiU beoooie inform 
mainly for an audience of pdicymakers at federal and perh^ state levels governance. Benefits to 
tethers arid adriiiriistratins are snoall if diere is any soise at all (tfbetiefits at the local leveL Local 
teadiers and administrators are lilKly to have little stake in ti» process that has been designed to 
generate indicator data, even though indicators themsdves have as naidi relevance to planning and 
p(4icymaking at the scbocd site as they do at natknal levels. 

Ownership o£ infcanatiai is linked to ownersh^ ci process and to the proximity to 
decisionmaking to affect the chc^ of values that indkattssonbody. Ownership (tf prwess gets 
establisted at the level of effort where problems in lemlersfaip have been well articulated and well 
heard, motivaticm to seek a sdution has reached a critical mass, and intdlectual capital has been 
invested in designing forms, surveys, assessmeitts, and soon diat will usually generate information 
that is thought to be needed. Teachers and administrators may help to collect indicator data. However, 
they have little s»ise o{ ownership and probacy no real understanding oi the strategics that transform 
terjad constructs (e.g., class size and teacher qualifications) into data that rqpresent gains losses in 
ichool quality. Mmeover, there is Utde sense of how policies made by teachers and adnunistrators in 
managing education at school sites affect values that indicators may take in the future. 

Costs are assessed against effort, even at the fiederal level of ownership, but diey are more likely to 
be sensed as an expense for overhead ratho" than an outlay for direct effort When the effort to 
generate information fi^ a syston of indicates is local, not federal, teacl^ and administrators assess 
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the OTSt of ^dwring it against its utility. If costs are perceived as nx> high, local teachers and 
adinimstxatDrs are xiKxrelikdy to show ledstamx to participate in data c^^ Indicators that 
require more effort from tead^xs and site administraiOTS are likely to be sensed by tlwse v/ho design 
data collection plans as having higher costs, even if no real dollar amounts are chai^ged against 
anybody's budget 

Perceived costs for some haxd-to-get-at indicattns may be mediated by re-establishing ownership. 
In diis case« owner^p of jaooess do^'t need to be shifted so much as it needs to be teoadened 
directly to local sites, rather than state cnt district sites. Paying local sites to generate information about 
context and jnocess indicatcns doera't^taUish ownership of process. It isn't sufl^ent to assure that 
infimnatiai will be (tfhi^ quality, ai^i it may ncsi be Mcessary. Afkir^sing the need of letters at 
local sites— dtter strong principals at tradid(Hially managed school sites or management teams at sites 
wl^re dedsionmaking is shared~is a bettn- bet 

School sites do engage in policymaking. Decisions about long-torn provision of school services 
are made at naticmal, ^a^ district and sdKX>l site levels. DifFer enoes among Aem relate nuare to scale 
dian to spheres of exclusive influemx. Even in situati(»is where schoding at a kxal site is affected by 
states federal legislation, long-range decisions of bow best to comply with state or federal regulations 
place principals (or mana^nent teams) in the role of fcsming policy. Hall et aL (1985, p.8) describes 
an educational system as an organizaticHi where resources are converted into educational services for 
pupils. Fn»n their perq)ectxvetimb]ic education is a nested set of educational systons that exists at the 
levels of classroom, school, district or state, each with varying respcmsilnlity for governance. 

Rationale for what an indicator system shmikl do are ccmstructed around needs-to-be-met that are 
as pressing at the local site as fl»y are at stats and federal levels. For exan^le, McDonnell (1987), in 
discussing the needs of policymakers versus researdwrs, loaintains that researchers want a ixiodd 
indicators that have theoretical relevance and causal inyact Ind i cator s should help researchers to 
identify factcns in educatkm sad how they interrelafe to produce effects. The policy ccHnmunity, 
however, is more conccn%d about indkators tlot ickntify fact(»s ihcy can influence—factors that 
aiq>ear to be preseiit when the directiimc^ a valued outcome is ccnsislmtly altered. Toe^)edfic 
intenelati(»shipsainong these £actois as they produce effects is less inqxmant than the consistKicy 
with which effects are produced when all are jHescnt 

School sites also have a stake in knowing more about dq)endable associatkms that exist between 
conditions of schooling and desired outcomes. School-site mana^rs are as involved as executives at 
more inclusive levels of edu^on in making dedsions having an uxqiact intended to be long term. In 
om way or axKXlwr, tl^e decisions are ccmsdous attempts to deal with resources and how they are to 
be used; delivery of services to diverse sut^xspukuioos of students; stamlards for what will be taught 
who will teach, and what students will be eligible to partknpate; goals in achievement that will be 
pursued; and outcomes that will be accepted. 
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Shavelson et aL (1987) idmtify five^^ fiinctioas that an indicator system might serve in the policy 
contexf of a naticmal system of indicators. Four of them are equally relevant to management and 
policymaking at schod sites: 

• Describing status, siKh as levd of i»nicii»tion in sciemx mathonatics by ethnicity, 
gender, and social class; 

• Piroviding an early warning by idoitifying enaerging trends and problems, such as sharp 
declines in achievement by certain subpO|nilations; 

• Idoitifying pdides that qq)ear ID be siwceedingfS'unintenitedconsequeiKres of p(^k^ 
that have been put into effiect; and 

• Supporting leadership in school rtficsin, sixh as information that shows to what extent 
students are engaged in "hands on" scioice. 

Lx>cal policymaking needs the same kind of data-driven enhancemoits as federal policymaking. 
Local principals and management teams may not be pressing fior education indicators with the same 
intensity as stale and fedend pcdicymaksrs, but ti»ir cdlective alence may have little to do with a lack 
of capacity to make an association b^wecn indicator data and long-term decisions. More hkely, site 
managers have not matte an issue of having better indk»iors because: 

• Schod sites have never had much data that pemined^)ecifically to the site; 

• Data that have been availaUe deal inainly with scores cm standardized achievement tests that 
are not linked very closely to other factors, such as background characteristics of students, 
quaHficatkms of teachers, and what teactos teach; and 

• Generation of data has seldwn beci under site ccmtroL Local sites tend to be givm data 
ratho'thanac^^tytogaieratedata. In (XherwiHds, they receive data that have been 
packaged for them. Furtfaemxae, the cycle in which data are provided may not match the 
cycle in wh^ site dedsions are made. 

Looking to the Future 

As iseviously mentioned, com|»ris(») of NAEP achievemoit data across sta^ will begin in 1991 with 
grade 8 mathematics perftaiance. As was the problon with the U.S. Department of Education's 
''Wall Chart," NAEP urhievement data are hard to disaggregate in ways that show fair coaq»ris(ms 
across states. The use of "co-statisiics," along with the NAEP achievement data, has been ;ffcqx>sed 




^^The five fioK^om m: describing n^kxal stains, describing sate oatus, early warning, inf(»ming policy snA i»actice, 
and {KovidiDg kadersh^ 
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by the DqyartiiKnt of Edi^atim as a way adjusting NAEP differences to make than more "fair". 
However, there is little evidence that the variables pnqxMed as "co-statistk^" are related to states' 
average achieverooiL These include (amraig others) per capita income, length of schod year, use of 
con^tency tests, average tead»r salary, and per pupil expenditure. According to Koretz (1991), 
adjusting for differences in state-level median income will not malce NAEP conqjarisons fair if, for 
exanyle, di ffe reiKCS in aate-levd median incOTie rqaesrot irrelevam difiFerences in the C(^ 

Legislation currently pn^bits use of NAEP to compare schools or districts, but efforts are mda 
way to introduce new legislation to allow this. The original mraitraing functioned NAEP has been 
stq}planted in recent years with acoountaUlity and explanatcay fumrtkxis, thus the resultant movonent 
toward distria-level NAEP. At the same time, several naticmal grraps, inchiding tte Natii»ial 
Educatira Goals Pand aiid cons(»tia of rNearchers and piactitKiners, are Khrocating a new national 
regional) test which is givra to all stuctents aiKi used for accc^mtaNlity purposes. Much support, 
e^jedally at the preademial level, has been generated for this idea. 

National efforts to develq) and cdlect indicator data may not cuzrendy be able to provide much 
indicates' information to local districts or schools, but there are ways in which the wealth is 
accumulating from this work can be distributed to local districts. Hrst, die NCES q>erates an annual 
naticmal survey, the Conmson Core (d Data (CX!D), whkh includes bonh fiscal and nonfiscal data. The 
survey collects data at the sdiool, distrkn and state level, but gaierally repom data cmly at tte district 
level or higher. Since 1988, a repon released annually describes the characteristics of the 100 largest 
puUic elementary and secondary school distrto in die United States, including such data as 
emoUmoits, number graduating students, number widi IiKlividualized Educational Plans (lEP), and 
ptqnl-teacher ratios. Recent i m pr ovemen ts in the CCD d ataba se, including agreement anxag states on 
common definitions and mohods, have ei»bted a toon ccm^sehenstve look at the nation 's and states' 
breaddi at id quality of schooling. In conjunction with tl» CCSSO, NCES and its national Foram 
group have focused in recent years on develqnng a comisehensive, accurate, and timely reporting 
system for delivery to states. 

In a^int NCES-Cmsus-CCSSO project, the 1990 C^isus napping currendy is being carried out, 
converting census blocks to the nation's 17,000 sclKXd districts. School district boundaries are being 
superin^TOsed on a census map by block and die infcamatkm digitized and converted to Census TIGER 
files. State co(»dinators assist in the tni^ing. More dian 200 tabulatkms covering donogra 
characteristics will be run at die schod district level and distributed free to districts in 1992, along widi 
a CD-ROM disc and user-fiiendly software. The CD-ROM file will include data from the 1990 
Census, along with die NCES CCD n># to allow merging of CCD (kta widi Cdisus data. Var^ 
include a large range of doxiographic population data, fiscal (^aracterisdcs, as weU as educ^tio^ 
ccmtext variables, such as dropouts and percentages d fiee lunches. It is likely that state level and 
district-level outcome (kta also can be added to die database to aUow analysis oi relationships. 
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As n^ock^ogical and dma (xUectimi jnoblran are solved at ^ f^tei level, local districts can 
bei^t by using those solutions to develop their own infoimation. Many of the problems being 
ackirKsed nationally in d» ctesign of data cdlectioos are similar to protdems that local sites face on a 
smallo' scale. For exana^jle, much (rf the ^fbn in naticmal projects go^inn>tl»des^ 
cdlection so that information about stiKtent outcomes car be disaggregated by ethnidty ami gender and 
by other variables that rq>rNent school background diaracteristics, schod resources, ami school 
processes. Local sites need to be able to see many of the same kinds of disaggregaticms if they are to 
do well in making good pdicy. 

New kiK>wledgB about indicates being genoated at the federal level that also can benefit local sites 
relates to research rpconmiwided to collect indicator data abomcont^ and i»X)^^ 
'Vhat is taught" and students* **oppoitumty to leam." This kind of research needs to be done in 
coUabcHation with teacho^ and administrators at local sit^ 

Involvement of local administrators and teadim in dtfferrat adjects of design and intcipretadon of 
data has two potential bo^ts. Fvsu thoe is a beginning of an investn^t in the most basic units of 
an infrasmKmire-local ^tes-for genmting infcomation about edwstion. Sites that wok 
collabc»:atively widi nati(mal re^arch proj«:ts on devek^nng new iiHlu^tors will become models for 
how lo(^scbo(d sites can geiKratequatityinfiDnmtionabomamtext and pnx:esses. Second, the 
methods and tedinobgy that grow out of these ]m)jects may be nwre feadble to carry 

om than f<x the staffof a nadcmwide project Lo^ sit^ do not assess costs in die sanw way as 
nadcHial projects. Whatisatiuecostof data cdlecdon for a naticnal project may be noore like staff 
development and ccdlabonoive planning when the data coUectioa is ''owned** by site-level staff. 
Agencies who sponsor or conduct researeh on new such indicates slKxild ccmsider ways to 
disseminate findings to local sites, even bef^ne some of the findings can be incoiporated into data 
coUectkm ^brts to siqyport a naticmai system 

Socoe of the problems associated with (tevelqnng a naticmal indicate system are not relevant to the 
problems that school sites would have in genoating indicator data. ProUems {^gaining coopoation 
by states, districts and schools also are not retevant 
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Of particular reIevaiK:e to local sites are problems such as the following: 

• Levelof detail of survey questions needsd to get at different a^)ect5 of an indicator 
variable; 

• Ccmsideraticms about consistency in data collection fpcHD one year to the next that will 
improve the comparatality of indicator statistics over time. In particular, what kinds of 
pfcMems need to be antK^mted in maintaining a san^le across years? 

• Mettods used » obtain different kinds of infonnatum that are critkalknow^ 
sites if iiKl^atcas generattd are to be meaningfully ccxnpared to natk>nal standards; 
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• Agreemem on ccmunon definitions of imjxirtant tenns such as dropout; and 

• Integration of data from varices sources. 

Although federal devekqmients may eventually enabte ^aies and distxkts to dther obtain relevant data 
or to develop their own indicator system, administratca' training and ^aff development in this field is 
not a federal resp(»isilnlity. Although it is becooiing iiKieasingly posaUe to envision a 
comprehenave d at a ba se in operation in perhaps 90% erf sdiool districts and in many sdKX)l ^ 
the year 2000, state agaides pn>baldy wiU need to be trained by fecteral-levd reiser 
such a database. In turn, state trainers couki train sta£f the district level, and school personnel could 
be trailed by district staff. A state comnitmait to bringing tte capacity to mcmitor changes over time 
and disaggregate to important oonstituoicies would be neoKsaxy. 

There is an inopotant need to ascertain the capstdty of sites to goierate indicator data to support 
theh'ownmanagemaiL First, althou^ iix)st schod adiniiiistrators have access to powerful computer 
hardware and software ai^lications (such as d-base and Excel), many use compute piimaiily for 
word processing. Easy-to-use technolc^ for data management abtady is in place, and the investment 
in training needed to take significant advantage of the available technology is fairiy modest Few local 
school sites are likely to be staffed in ways that wouM suppc»t on-site generaticm of indicatcns. 
However, the current lack of staffing is likely to have more to (k) with how the tasks of school 
inariagemait have evdved and how tiine is allocated than it has to do with lack of interest or talent As 
schools across the country continue a process erf restructuring, the kxnis of responsibility for school 
managemoit and policymaking shifts closer to school sites. The roles of principals and teachers no 
longCT fit naore classic roles of management and labor. Increased responsibility for school site 
managen^ brings increased oppoftunities for different kinds of staffing. Tte £ict tiiat school sites 

have not engaged in serk>us effcnts to geittrate data and maintain databases in support of poUcymak^ 
should not tiiat they will not develop this kind of capacity in die future. The times are changing. 
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